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ABSTRACT 

This study examined the National Board for Professional 
Teaching Standards (NBPTS) certification process, investigating 65 white and 
African American teachers from North Carolina, Ohio, and the District of 
Columbia who had sought National Board certification in one of two areas— 
early adolescence/English language Arts and middle childhood/generalist. Of 
the 65 teachers, 31 achieved certification. Their certification scores fell 
into one of four groups: well above or below the performance standard or just 
slightly above or below the performance standard. Researchers observed the 
teachers, interviewed teachers and their students, administered student 
surveys, and collected students 1 written responses to a statement. All 65 
teachers demonstrated laudable teaching skills, but National Board certified 
teachers outperformed non-certif ied teachers on all 13 generally recognized 
measures of good teaching. Differences were dramatic in 11 areas. Certified 
teachers were better at improvising when faced with the unexpected, 
understanding why students succeeded or failed on specific tasks, engaging 
students without overwhelming them, and anticipating difficulties students 
might have with new concepts. Students of certified teachers demonstrated a 
deeper understanding of the concepts they were taught than did students of 
non-certif ied teachers. (SM) 
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the teachers 



I Measuring Excellence 

| What do National Board Certified Teachers have in common with Olympic athletes? Both 
| represent an elite class. Through innate ability and dedication to their craft, and to their own 
improvement, Olympic athletes have distinguished themselves from a field that includes 
many other excellent competitors. The same is true for National Board Certified Teachers, 
who go the extra mile to help their students achieve at higher levels. 




Like Olympic judges, the National Board for Professional Teaching Standards (NBPTS) 
recognizes experts. Teachers who earn the Boards national, advanced certification show 
they meet high standards in a rigorous evaluation that examines their knowledge of chil- 
dren and subject matter, and their ability to put that knowledge 
to use effectively. 

But measuring accomplished teaching is complicated. Unlike a race 
that determines an undisputed winner, NBPTS certification requires 
the application of professional judgment. Candidates are judged by 
their peers on standards that leading educators and scholars have 
determined are essential to learning. This type of jury system has 
been used for decades in other fields, such as art and architecture, 
but it is not common to professionals in elementary and secondary 
education. NBPTS introduced national certification of these teach- 
ers less than 10 years ago, and because it was new and unfamiliar, 
some in the education community were skeptical. The Board sets the standards, the Board 
oversees the assessment. Is there really a measurable difference between teachers certified by 
NBPTS and others who did not meet its standards? 



The answer is a clear yes, according to a bold new study of the National Board Certification 
process. NBPTS asked a team of researchers to take an unblinking look at the certification 
system to ensure it is doing what it claims to do — recognize highly accomplished profession- 
als. No other profession with a national process for certification has subjected itself to such a 
review and made the results public. In the end, the study provides evidence to support what 
teachers and policymakers have said from the outset — National Board Certification is a dis- 
tinction that matters. 



The Study Group 

The research team, led by Lloyd Bond of the University of North Carolina at Greensboro, 
looked at 65 teachers from North Carolina, Ohio, and the Washington, DC, metropolitan area. 
The sample group was recruited from a pool of eligible participants who met these criteria: 



0 

ERiC 



• They had sought National Board Certification in one of two areas — Early 
Adolescence (ages 1 1— 1 5) /English Language Arts and Middle Childhood (ages 
7-12)/Generalist. Why these areas? The researchers wanted to get a feel for whether the 
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effects they were looking for could be found across student age groups and teacher certifi- 
cate categories. Thirty-one of the 65 teachers achieved certification, but that does not 
mean the team compared outstanding teachers to lower achievers. The non-certified teach- 
ers were highly competent professionals who in many cases had just missed the standard 
for certification. 

• Their race/ethnicity was white (non-Hispanic) or African-American. Teachers of other 
ethnicities have not applied for certification in large enough numbers to be effectively 
studied. 

• Their National Board Certification scores fell into one of four groups — well above the per- 
formance standard, well below the performance standard, and just slightly above and 
below the performance standard. Its important to note that although the directors of the 
research effort knew participants certification status in order to recruit them for the study, 
the teams of teachers that observed and evaluated the study participants did not. 



The Study Participants 

Average Years’ Teaching Experience: 15 



Race Certification Area 




NOTE: Researchers found no correlation between the two study groups and the socioeconomic 
status (SES) of their students. 




Key Findings 

• All 65 teachers in the study demonstrated laudable teaching skills, but the National Board 
Certified Teachers outperformed their non-certified counterparts on every one of 13 general- 
ly recognized measures of good teaching, and the differences were dramatic on 1 1 of them. 



• For example, the National Board Certified Teachers were better at improvising 
when faced with the unexpected, understanding why students succeed and fail 
on a given academic task, engaging students without overwhelming them, and 
anticipating difficulties students might have with new concepts. 

• Students of the National Board Certified Teachers demonstrated a deeper 

understanding of the concepts they were taught (in units the researchers had 
observed). Nearly three-quarters of the National Board Certified Teachers pro- 
duced students whose work reflected deep understanding of the subject being 
studied compared with less than one-third of non-certified teachers. *See pages 
1 0—1 1 for more details on the study results. n 






the criteria 



Attributes of Exemplary Teachers 

The researchers began the study by analyzing the extensive body of published studies 
on expert teaching. From this review of the literature, they identified 13 key attributes 
that surfaced in study after study that characterizes expert 
teaching. The research team based its evaluation of the 
teachers in this study on the following 13 attributes of 
exemplary teaching: 

Use of Knowledge — Displaying appropriately deep 
knowledge of a subject, constantly adding to that body of 
knowledge, and connecting new information with other topics in 
the curriculum to help students delve below surface understanding 

Deep Representations — Drawing upon a vast array of 
materials, assignments and approaches to provide tailored 
instruction for students with varied backgrounds and 
abilities 

Problem Solving — Defining and refining instructional 
and curriculum issues to reach insightful solutions 

Improvisation — Adapting and modifying instruction during the flow of lessons as the 
situation dictates 

Classroom Climate — Creating optimal environments for 
learning, in part by becoming acutely aware of how stu- 
dents are responding to instruction 

Multidimensional Perception — Displaying a high 
level of awareness by reading verbal and nonverbal cues, 
filtering relevant information and identifying priorities for 
action 









worked in 



Sensitivity to Context — Appreciating the 
essential characteristics of students and 
classes and using them to guide decisions 

Mon/tor/ng Learning and Providing 
Feedback — Keeping a careful watch on 
how well students are responding to les- 
sons and assignments and offering con- 
structive criticism of their progress 



Test Hypotheses — Reflecting on how well ideas 
practice and refining future lessons accordingly 



Passion forTeaching and Learning — Expressing a high level of emotion over suc- 
cesses and failures and inspiring students to become more excited about learning 



Respect for Students — Displaying not only a high regard for students’ learning and 
well-being but also helping them find ways to overcome barriers to learning 

Challenge — Articulating high expectations and formulating lessons and activities that 
are more demanding and engaging for all students 



Deep Understanding — Attempting to provoke curiosity and developing inquiry and 
problem-solving abilities in students by thoroughly examining the intricacies of a 
problem or issue 



It was necessary for the research team to evaluate each of these attributes individually 
within each teachers practice to study and compare them effectively, 
but expert teaching really is more than the sum of these indi- 
vidual traits. These teachers know their subject and they 
know education, and they constantly draw upon that 
knowledge to understand where students are, where 
they need to go, what work to put in front of them, 
how they spend their time, who needs more and less 
attention, and how to address individuals and groups. In 
the end, accomplished teaching is the successful blend of all 
these attributes. ^ 




the method 



Collecting the Data 

Once the criteria were set, the research team designed a sophisticated system that would 
allow teachers in the study to be evaluated fairly and objectively. It began with one teams 
systematic gathering of data that a different team would later assess. 




The Observers 

Who They Were: Twenty-eight educators, the majority of whom were 
classroom teachers with an average of 25 years’ experience in a variety of sub- 
jects and grades. Many had received awards that recognized their teaching 
and other contributions to education. 



Race 



Gender 





Responsibilities: After three days of rigorous training, the observers were grouped in 
pairs and assigned teachers to visit. Their role was solely to collect information according 
to strict protocols, not make judgments. During the observation, they knew only the 
teachers name, location, grade level, and subject matter. They were not told of the 
teachers certification status and agreed not to seek it out. 

What They Did: Observed the teachers in their classrooms to witness them teaching 
one lesson within the larger unit of instruction that they were focusing on for the study. 
As one observer watched the teacher to study how he or she explained the material and 
responded to students, both verbally and otherwise, the other completed a form designed 
to record classroom interactions. An observation protocol guided the team to ensure 
each one looked at the same attributes (directions and procedures, monitoring, feedback, 
management, adaptation, questioning) in the same way (by taking detailed notes and 
recording nonverbal cues). 

After Class: The observers interviewed, separately, three students randomly selected 
by the researchers for 10 minutes each in a nearby classroom or other private area. The 
line of questioning was aimed at determining how well the students understood the 
days lesson. The pair then interviewed the teacher for about an hour and left a packet 
containing questionnaires, and a writing exercise for the students, instructions for col- 
lecting additional student work samples, and an addressed, stamped packet for the 
return of those materials to the researchers. H 






The Casebook 



** The researchers compiled a casebook of all 
Each casebook contained: 



the materials collected for each teacher. 



• An. explanation of the unit being studied 

• Transcripts of the tape-recorded lessons arid teacher 
and student interviews (with identifying information 
marked out) 

• Copies of all unit-related work completed by four 
randomly selected students 

• Completed writing exercises designed by the research 
team and assigned to students 

• Completed student and teacher questionnaires 
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To measure their motivation and independence, students were given two surveys 
crafted by the research team. Here are just a few sample questions from the first 
one, “About My Schoolwork and About My Class.” Students were asked to mark 
whether they strongly disagreed, mostly disagreed, disagreed, agreed, or mosdy 
agreed with statements such as these: ■> 

• An important reason I do my schoolwork is because I know we will get 

graded on it. , . i 

• I want to get right answers, even if I don’t understand the work. jj 

• I like schoolwork that I’ll learn from even if I make a lot of mistakes. | 

• I like schoolwork best when it really makes me think. | 

• I can do almost all the work in class if I don’t give up. jj 

• No matter how hard I try, there is some class work I’ll never understand. | 

• When problems arise in class, they are dealt with fairly. | 

• The teacher explains what will happen if someone breaks a rule. 



Teachers in the study also agreed to 
ask their students to spend 50 to 
55 minutes writing a response to a 
statement. They were not given 
time to prepare. The writing sam- 
ples were returned to the research 
team. Here is an example of a 
4 prompt” given to students in 
grades 4 to 8. 



EXAMPLE: 

Grades 4 to 8 Writing Prompt 

Imagine that your school is 
planning to require new school 
uniforms. All students would be 
required to wear tan pants , 
shorts , or skirts but could 
choose red [ white, or blue 
shirts. Write a letter to your 
principal explaining why this is 
or is not a good idea. 
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the analysis 



Evaluating the Data 

Completed casebooks went to a separate team of assessors, which reviewed them to find 
evidence of the 13 attributes of teaching expertise that were used to compare National 
Board Certified Teachers and non-National Board Certified Teachers. 



The Assessors 

Who They Were: Twenty-two National Board Certified Teachers from North 
Carolina and doctoral students in .the field of curriculum and teaching. Team members 
averaged 17 years’ teaching experience in a variety of subjects and grades, and more than 
half had advanced degrees in education. 





What They Did: After rigorous training to guard against bias that can creep into peer 
evaluations, plus in-depth instruction on the individual attribute they would be scoring, 
the assessors began their work. They reviewed randomly assigned, numbered casebooks 
cleared of all identifying information about the teacher. Using a four-point scoring 
guide, or rubric, they focused on all evidence throughout the casebook that related to the 
particular attribute they were to evaluate. Each assessor assigned a score, from 1 to 4, 
that he or she thought best reflected the value of the evidence collected. (See sample 
rubric on facing page.) Scores were noted on a paper form and online. 

How the Scoring System Worked: Each of die 13 attributes had its own scoring 
rubric, which consisted of a broad explanation of the attribute, and specific examples of 
behaviors and knowledge teachers would typically display at both the novice (1) and 
expert (4) levels. Scorers compared the rubrics with the evidence in the casebooks and 
rated each teacher accordingly. For example, the casebook of a teacher who scored a 4 on 
a particular attribute contained abundant, convincing and sustained evidence that showed 
she met the rubric’s standard. The casebook of a teacher who scored a 1 contained scanty, 
or even negative, evidence of performance on that attribute. Each casebook was independ- 
ently reviewed and scored by two assessors. Q QQpy AVAILABU 
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Sample Rubric 


Improvisation 




Experts adapt and modify their instruction during the flow of lessons. While competent 
teachers may make adjustments to their plans over time, experts are more likely to 


assess the impact of their instruction on an ongoing basis and make changes on the 
spot. Experts draw upon an extensive repertoire of patterns of action while teaching, 
incorporating them into instruction that is continually responsive to students. 


Level 






A teacher • Designs and implements 


• Makes immediate and 


• Responds readily and 


at this level: opportunities for “on-the- 


instructional^ effective 


effectively to things that 


spot” learning 


decisions during the flow 


students say and do and 


of lessons 


adjusts plans accordingly 


Level 3 • Develops plans that allow 


• Makes some instruc- 


• Alters instruction based 


A teacher for some flexibility as 


tional-effective decisions 


on general patterns of 


at this level: lesson progresses 


during the flow of lessons 


student response 


Level 2 • Develops plans that are 


♦ Follows predetermined 


♦ Alters instruction only 


A teacher specific and allow for 


lesson sequences with 


when confronted with 


at this level: little flexibility as lesson 


little variation 


unexpected, major 


progresses 




challenges to prespecified 
lesson plan 


Level 1 • Rarely, if ever, develops 


• Experiences confusion 


• Resists any alteration in 


A teacher plans that allow for 


and frustration when stu- 


instruction 


at this level: flexibility as lesson 


dents respond or events 




progresses 


occur in unexpected ways 





Determining Final Scores: When the asses- 
sors’ scores disagreed, a lead assessor helped them 
resolve their differences in conference. At no time 



did the lead assessor attempt to sway either of the 



original assessors. 





the results 



Teaching Practice 

Although the study’s sample size was relatively small, it produced a vast amount of 
data that supports NBPTS’s contention that it is, in fact, identifying expert teachers. 
The 31 National Board Certified Teachers in the study, as a group, consistently 
demonstrated a higher level of expertise on all 13 attributes measured in the study. 
While differences in average scores for two of the attributes — Multi-dimensional 
Perception and Monitoring Learning — were too small to be called statistically sig- 
nificant, the differences on the remaining 1 1 attributes were indeed impressive. 
Here’s how the two groups compared: 



Average Scores for Board-Certified and Non-Board-Certified Teachers 



Mean Score 



4 . 0 - 



3.5 - 



3.0 - 



2.5 - 



2.0 
1.5 - 
1.0 



■ Board-Certified Teachers 

■ Non-Board-Certified Teachers 




Climate 



Deep 

Understanding 



Monitor/ 

Feedback 



rassion i respect" "I' “lest 

Hypotheses 



Challenge Deep Improvi- Multi- Problem 

Representation sation dimensional Solving 



Sensitivity Use of 

Knowledge 



This evidence is all the more compelling because of the size of the study group. The 
smaller the sample for a study, the harder it is to find statistically significant differences 
among participants in a comparison such as this. For this reason the research team 
believes these results are especially strong. 




Student Outcomes j j 

The researchers didn’t want the study to rest entirely on the comparison of teaching prac- 
tice alone. Each attribute of accomplished teaching ultimately is tied to student outcomes. 
Although the connection between strong teaching and successful learning is notoriously 

BEST COPY AVAILABLE 



difficult to measure, the researchers did 
want to see whether the students of 
National Board Certified Teachers 100 

achieved at higher levels than students 
r i * 80 

of the comparison group. 

i 60 

To measure this, the researchers 
reviewed completed assignments of 4 Q 

the four students they randomly chose 
from the roster of each teacher. 20 

Collecting the work proved especially 
challenging. Ten of the teachers did not ® 

produce samples despite repeated pleas 
from the researchers, and samples from 
nearly half the teachers were not usable. Still, the teams were able to cull comparable stu- 
dent work samples from 36 teachers (19 certified and 17 non-certified). Their two-stage 
analysis, completed with the well-established Structure Of Learning Objectives (SOLO) 
rubric, found 74 percent of the Board-certified teachers produced students who demon- 
strated “deep understanding” of the unit they were taught compared with only 29 percent 
of non-Board-certified teachers. Reviewing work that ranged from completed worksheets 
to papers to “artifacts,” such as completed presentations or products, the assessors looked 
for evidence of understanding according to SOLO’s four levels: 

• Unistructural — Students understand one aspect of the assignment 

• Multistructural — Students understand more than one aspect of the assignment, but 
the facts are not interrelated 

• Relational — Students understand several aspects of the assignment and tie them 
together 

• Extended Abstract — Students are able to relate these multiple concepts and generalize 
them in ways that were not covered in class 

The assessors labeled work samples that fell into the first two categories “surface” and 
work that fell into the latter two categories “deep.” 

These findings, especially if confirmed with larger samples in the future, suggest NBPTS 
is identifying teachers whose students gain a solid grasp of the subjects they are taught. 



Students’ Level of 
Understanding 



■ Deep Understanding j 

■ Surface Understanding 




Student Samples Student Samples 

of NBCTs ofnon-NBCTs 



A Final Note 

This is just the first of several planned studies of the NBPTS certification process. The 
research will be continued in other certification areas and with larger study samples. 
Nevertheless, it is an encouraging sign that the first study produced strong evidence of 
the certifications validity. And it sends a clear message: When states and local school dis- 
tricts encourage teachers to practice at this level, children benefit. 
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NBPTS at a Glance 



Founded: 1987 

Organization: Nonprofit, nonpartisan, independently governed 
by 63 directors (the majority of whom are classroom teachers) 

Mission: Maintaining high and rigorous standards for what 
accomplished teachers should know and be able to do, providing 
a national voluntary system certifying teachers who meet these 
standards, and advocating related education reforms to integrate 
National Board Certification in American education and to 
capitalize on the expertise of National Board Certified Teachers 

Certificate areas: 26, including generalist and subject-specific 
certificates for early childhood (ages 3-8) through young adult 
(14-18+) levels; more than 30 planned 

Certification process: During the school year teacher candidates 
assemble a portfolio that includes, among other things, videotapes 
of classroom interactions and samples of student work plus the 
teachers reflective commentary on the evidence. Candidates also 
document their education-related work outside the classroom. 
They demonstrate their knowledge of teaching generally, and their 
subject area specifically, at a one-day assessment center. Trained 
NBPTS evaluators who teach in the same field as the candidate 
judge all components of assessments. 

Number of National Board Certified Teachers: 

More than 16,000 in 24 certificates areas 

Number seeking certification as of January 2002: 

Approximately 21,000 



States that offer incentives for teachers 
to pursue certification 




■ Support National Board Certification 
with Incentives for Teachers 



The vast majority of states and local districts in the U.S. support 
National Board Certified Teachers in a variety of ways, such as full 
payment or subsidies of the $2,300 certification fee, one-time grants, 
annual bonuses, salary increases, extended licensure, release time, 
professional development credits, and license portability. 
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Contact information: 1523 Wilson Boulevard, Suite 500, 

Arlington, VA 22209; 1-800-22-TEACH; www.nbpts.org 

© Copyright 2000, National Board for Professional Teaching Standards. This project 
is funded in part with grants from the U.S. Department of Education and the 
National Science Foundation. Through September 2001, NBPTS has been 
appropriated federal funds of $109.3 million, representing approximately 
31 percent of the National Board Certification project. More than 
$l06.3million (49 percent) of the projects cost will be financed 
by non-governmental sources. The contents of this report were 
developed under a grant from the Department of Education. 

However, those contents do not necessarily represent the policy 

of the Department of Education, and you should not assume endorsement 

by the Federal government. 

This report is a synthesis of findings from The Certification System 
of the National Board for Professional Teaching Standards: 

A Construct and Consequential Validity Study. Its authors are: 

Lloyd Bond, Ph.D., University of North Carolina at Greensboro 
Tracy W. Smith, Ph.D., Appalachian State University 
Wanda K. Baker, M.Ed., University of North Carolina at Greensboro 
John A. Hattie, Ph.D., University of Auckland 

The study was published in September 2000 by the 
Center for Educational Research and Evaluation, 

University of North Carolina at Greensboro. 

For copies of the full report, contact 
O _ iPTS at 1-800-22-TEACH. 
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